The technology described herein relates to data processing systems and in particular to display controllers for data processing systems.
FIG. 1 shows an example data processing system that comprises a central processing unit (CPU) 7, a graphics processing unit (GPU) 2, a video codec 1, a display controller 5, and a memory controller 8. As shown in FIG. 1, these units communicate via an interconnect 9 and have access to off-chip memory 3. In use of this system the GPU 2, video codec 1 and/or CPU 7 will generate surfaces (images) to be displayed and store them, via the memory controller 8, in respective frame buffers in the off-chip memory 3. The display controller 5 will then read those surfaces as input layers from the frame buffers in the off-chip memory 3 via the memory controller 8, process the input surfaces appropriately and send them to a display 4 for display.
FIG. 2 shows an example data path for the processing of the input surfaces for display in the display controller 5. It is assumed in this example that the display controller 5 can take as inputs for a given output surface to be displayed a plurality of input surfaces (layers), and includes, inter alia, a composition engine (stage) 22 that is able to compose one or more input surfaces (layers) (e.g. generated by the GPU 2 and/or video codec 1) to provide a composited output frame for display.
As shown in FIG. 2, the display controller includes a DMA (Direct Memory Access) read unit 20 that reads data of input surfaces to be displayed and provides it appropriately to local buffers of the display controller, in the form of respective sets of latency “hiding” FIFOs 21. (The latency hiding FIFOs 21 provide “latency” buffering in the display processing path to allow for potential latency in retrieving the required input surface data from memory. There is one set of latency FIFOs 21 for each “layer” that the display controller can take as an input for its processing.)
The input surfaces that the display controller 5 processes to provide the output surface for display will be generated, as discussed above, e.g. by the video codec 1, CPU 7 and/or GPU 2 of the overall data processing system, and stored as respective frame buffers in the main memory 3 of the data processing system.
When a frame is to be displayed, the input surfaces that form the input layers are composed in the display composition stage 22 to provide a composited output surface for display. The composited output surface (i.e. the frame that is to be displayed) is then subject to display timing control 23 (e.g. the inclusion of appropriate horizontal and vertical blanking periods), and then provided to the display output interface of the display controller 5 for provision to the display 4 for display.
This process is repeated for each frame that needs to be displayed, e.g. at a rate of 30 or 60 frames per second.
As such display processing is a real-time operation, the display controller 5 needs to deliver the pixel data to be displayed to the display 4 (to the display output) regularly, in each clock cycle triggering the display output from the display controller.
Thus a generally desirable feature of the operation of a display controller when displaying a frame is the ability to fetch the data for the layer or layers making up the frame to be displayed, such that the display can be kept updated in real time with new data at the required rate. If the data required to be displayed cannot be fetched sufficiently quickly, then there may be no current data available for display, such that an error will then occur. This is often referred to “underrun”, and will result in incorrect data being displayed.
Such “under-run” can occur, for example, because of latencies in fetching the input surface data from memory, such that the required data has not been fetched and/or has not completed its processing, by the time it is required to be displayed.
It is accordingly known to try to design display controllers and data processing systems to reduce or avoid the risk of underrun. For example, display controllers may comprise, as discussed above, sets of “latency hiding” buffers into which data to be displayed is fetched in advance of displaying that data. This can help to ensure that there is always sufficient data “local” to the display controller for display, even if there are delays in fetching data from main memory where it is stored.
It is also known to try to reduce the amount of data that must be fetched for displaying a given frame, for example by “pre-compositing” (“flattening”) a number of layers to be displayed (e.g. using a graphics processing unit), in advance of providing the layers to the display controller for display. However, this can increase power consumption, memory bandwidth and loading on the GPU.
It is also known to try to prioritise memory transactions for display controller operations to try to ensure that the required data for display will be available to the display controller when it is required.
Notwithstanding these techniques, the Applicants believe that there remains scope for improvements to the operation of display controllers and data processing systems, in particular in relation to the issue of “underrun”.
Like reference numerals are used for like components throughout the drawings, where appropriate.